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ABSTRACT 

The Elementary Technology Demonstration Schools 
program, where four elementary schools were equipped with computer 
hardware and software, was made possible by grants from IBM and 
Apple, Inc. The goals of the program were, in 3 years, to reduce by 
507. the number of stuf^ents not in their age appropriate grade level 
and those students not achieving on grade level in the areas of 
reading, writing, and mathematics; develop a comprehensive teacher 
training program for effective implementation and classroom use of ^ 
technology; and demonstrate to the community the educational benefits 
of technology. This report uses ROPE (Report on Program 
Effectiveness) comparisons, ROSE (Report on School Effectiveness) 
scores and comparisons, and standardized test results to describe 
student achievement after the second year of implementation and the 
first full year of computer usage by the students. A description of 
project implementation includes logs of computer use and employee 
survey responses as well as discussions of campus computer issues, 
training, parent take~home computers, telephone installation, and 
restructuring issues. Noting that the program has not yet provided 
the benefits envisioned, the report suggests that a long term view 
may be necessary to judge program effects adequately. Recommendations 
are given based on the findings of this report. Six appendixes 
contain the statistical data collected and 20 figures illustrate the 
findings. An executive summary of the report is also provided. 
(Contains A references.) (aLF) 
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Ba ckground 

This is the second year of the Elementary 
Technology Demonstration Schools 
program and the first full year of 
computer usage by the students. Tl\is 
program was made possible by a $44 
million dollar grant from IBM and a 
$74,000 grant from Apple, Inc. These 
grant monies were used to equip four 
elementary schools with computer 
hardware and software. The program 
proposal contains four specific goals. 

1. In three years, reduce by 50% the 
number of students who are not in 
their age appropriate grade level. 

2. In three years, reduce by 50% the 
number of students who are not 
achieving on grade level in 
reading, writing, and mathematics. 

3. Develop a comprehensive teacher 
training program to ensure 
effective implementation and 
classroom use of technology. 

4. Demonstrate to the community the 
educational benefits of technology, 
thereby obtaining support for 
districtwide implementation. 

Major Findings 

At this point in the Elementary Technol- 
ogy Demonstration Schools program, 
technology in the classroom has not yet 
provided the benefits envisioned. The 
evidence is not clear on the project's long- 
term effects, but some trends are begin- 
ning to emerge. 

1. Student achievement results, as 
measured by the Report on 
Program Effectiveness (ROPE) and 
the Report on School Effectiveness 
(ROSE), are mixed. Of students at 
the school three consecutive years, 
3 ROPE comparisons out of 24 
exceed the predicted gadn and one 
score is below the predicted gain. 
ROSE results of all students 
indicate 5 out of 40 comparisons 
exceed the predicted gain and six 
are below the predicted gain (pp. 3- 
9). 

2. ROPE results for summer school 
students show that out of 38 
comparisons, zero exceed the 
predicted gain and seven are below 
the predicted gain (pp. 15-17). 



3. Portions of the project are not fully 
implemented. Logged computer 
time is low at all three IBM 
campuses. Some teachers still resist 
implementing the Teaching and 
teaming with Technology instruc- 
tional delivery system. Telephones 
are not yet installed. One in five 
teachers say follow-up training is 
inadequate (pp. 17-25). 

4. The percent of overage students 
has decreased in some cases due to 
increased enrollment, but the 
number overage has not decreased 
(pp. 26 and 36). 

5. The number of students below the 
30th percentile in reading and 
mathematics has increased. The 
number of students failing the 
reading, mathematics, or writing 
section of the TEAMS/TAAS is 
increasing (pp. 27-28). 

6. The number of at-risk students is 
increasing (p. 37). 

From this short-term perspective, it 
appears students at the Apple school are 
impro'/ing achievement more than 
students at the IBM schools. The ap- 
proach at the Apple school relies almost 
exclusively on pull-out instruction in 
laboratories, which has aided earlier 
implementation. The approach at the 
IBM schools, by placing computers in 
classrooms as well as laboratories, 
recuires classroom restructuring. The 
demands of restructuring have delayed 
complete implementation. A longer-term 
view is necessary to judge program 
effects adequately. 



A copy of the full report for which this is the 
Executive Summary is available as Publica- 
tion Number 9130 from: 

Austin Independent School District 

Office of Research and Evaluation 

nil West 6th Street 

Austin, Texas 78703 

(512)499-1701 



Recommendations 

Based on the findings contained in this 
report, we make the following recom- 
mendations. 

1 . The computer technology needs to 
be fully implemented. Full imple- 
mentation includes all students 
using the computers in an instruc- 
tional setting on a regular, daily 
basis for em amount of time 
specified by the school. 

2. All schools need to set time-on-task 
goals and monitor their comple- 
tion. Where such goals do not exist, 
principals need to lead the staff in 
setting the goals. 

3. Teachers need to use the comput- 
ers in instructional activities. Use 
of the computers as a reward or a 
discipline tool needs to be elimi- 
nated. 

4. The logging system at the IBM 
schools needs to function accu- 
rately. All technical and user 
problems need to be resolved. 

5. The amount of follow-up training 
given to teachers needs to be 
increased. This will aid overall 
project implementation. 

Budget Implications 

Mandate: 

Required by School Board 
Funding Amount: 
Summer School $50,000 
Training N/A 
Maintenance N/A 
Funding Source: 
Local and external (private) 
Implications: 

The District is bound by an agreement 
with the two major fund providers of this 
project, IBM and Apple, to continue 
supporting the project and to .continue 
evaluation. As the District exmines 
ways to use State and local money for 
technology, the insights gained from the 
technology strategies employed in this 
project will be vital. 
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PROGRAM EFFECTIVENESS CHART 



EFFECT 


COST 


PROGRAM 


0 


$$ 


Elementary Technology 
Demonstration Schools 



Effect IS expressed as contributing to any of the 5 AISD 
strategic objectives. 
+ Positive, needs to be maintnined or expanded 

O Not significant, needs to be improved and modified 

Negative, needs major modification or replacement 
HiANi; Unknown, may have positive or negative impact 

on other iridicators; however, impact on the five AISD 
stiategic objectives is unknown. 



Cost i« the tjxpenso over the regular District per-student 
pxpondituro. 

0 No cost or minimel cost 

$ Indirect costs and overhead, but no separate budget 
$$ Some direct costs, but under $500 per student 
$ $ $ Major direct costs for toechers, staff, and/or 
equipment in the range of $500 per student or more 
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OPEN LETTER 



The overall goal of the Elementary Technology Demonstration Schools project : lu accelerate the 
learning of low achieving at-risk students and to enhance the learning of high achiving students so that 
all students are functioning at or beyond age appropriate grade level. Technology is one tool to attain this 
goal. While technology can be used for improvement in other areas, including acquisition of other skills 
(e.g., problem-solving and keyboarding skills), improving discipline and attendance, and enhancing self 
esteem, these are not included in the Project A+ goals. The project goals clearly state the aim of tlie 
technology in the context of Project A + , and the project must be evaluated in relation to these goals. 

Schools involved in the project do not use uniform methods. Schools employ two separate and distinct 
approaches to integrating technology into instmctional activities. This crucial point should not be 
overlooked when examining the project, just as the particularities of each approach should not be 
overlooked when judging the program's effectiveness. Other differences exist between the two 
approaches, most importantly the software and the software management system, but this report focuses 
on the approach used to integrate computers into instruction. 

Galindo pursues a strategy of placing computers in laboratories. Students are pulled out of the classroom 
setting, much like they are pulled out for art, music, and physical education activities, to work on the 
computers. The classroom is unaltered, and the computers are used primarily for supporting existing 
curriculum. 

Three schools, Andrews, Langford, and Patten, pursue a mixed approach that includes placing computers 
in tlie classroom in addition to laboratories. The project design calls for the classroom computers to be 
integrated into instruction through a centers-based approach, where groups of four to five students rotate 
through several learning stations, one of which is computers. Teachers may use the computers primarily 
to support existing cui. .culum, or they may choose to use the computers in a lead role. This more 
complex approach raises the demand on teachers, at least in tlie short run, and requires their willingness 
to become involved with changes in the classroom. 

The mixed approach is more difficult to implement. Placing computers in the classroom and integrating 
them through a centers-based approach necessitates a restructuring of the classroom environment from 
teacher centered to student centered. The new demands on teachers have been met with enthusiasm in 
many cases, with resistance in some. This resistance to the new methods is understandable; teachers 
learned whole-group instruction methods and have used them for many years. Nonetheless, this resistance 
inhibits complete implementation and obstructs proper evaluation of the two approaches. 

Given the differences between the two instructional technology approaches, and the increased complexity 
of implementing the mixed approach, it is not surprising that two years into the project students at 
Galindo are improving achievement more than students at the other schools. This conclusion at the end 
of a relatively short time span should not invite tjie conclusion that the laboratory-only approach is better 
tfian the mixed approach. However, it remains to be seen if the schools and teachers can successfully 
implement the mixed approach. Continuing evaluation and study are needed to determine long-term 
effects. 
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INTRODUCTION 



The Elementary Technology Demonstration Schools (ETDS) program of Project A -h began in the Austin 
Independent School District (AISD) in the 1990-91 school year. This is the second evalution report for 
the project. See Project A ^- Elementary Technology Demonstration Schools, 1990''9l: The First Year 
(ORE Publication No. 90.32) for evaluation of the first year of the program. 

Project A + , an AISD/IBM initiative established in the spring of 1989, is designed to improve the 
District's educational environment by acting as a catalyst for change and marshalling community 
resources. ETDS was made possible by a $4.4 million grant from IBM and a $74,700 grant from Apple, 
Inc. These grants were used to equip four elementary schools with computer hardware and educational 
software. The primary purpose of ETDS is to demonstrate the effectiveness of technology in 
accelerating the learning of low achieving at-risk students and enhancing the education of high 
achieving students. 

The project plan for ETDS spells out four specific goals for the computer technology program. These 
form the basis for the evaluation of the project contained in this report. Figure 1 displays the goals and 
how progress toward their achievement is measured. 



Figure J 

ETDS Goals and Effectiveness Measures 



project Goal 


Measure of Effectiveness 


In three years, reduce by 50% 
the number of students who are 
not in their age appropriate 
grade level 


Number of students overage 
one or more years 


In three years, reduce by 50% 
the nunnber of students who are 
not achieving on grade level in 
reading, writing, and 
mathematics 


1) Number of students below 
30th percentile in reading and 
mathematics on the ITBS/NAPT 

2) Number of students failing 
reading, mathematics, or writing 
section of TEAMS/TAAS 


Develop a comprehensive 
teacher training program to 
ensure effective implementation 
and classroom use of 
technology 


1) Employee surveys 

2) Teacher interviews 


Demonstrate to the community 
the educational benefits of 
technology, thereby obtaining 
support for districtwide 
implementation 


Degree to which project is fully 
implemented and its goals are 
reached 
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Last year's evaluation report contained only preliminary results regarding the impact of the project on 
student achievement because the computers were not fully networked i:ritil midway through the year. The 
report presented baseline achievement data and detailed the implementation process. 

This year's evaluation report is the first report following a full year of computer usage by the students. 
The report begins with a section on student achievement at the schools. It presents achievement data from 
several angles, including ROPE/ROSE, TAAS/TEAMS, and ITBS/NAPT.This section includes a review 
of the achievement of students involved in the summer school program. The report then examines the 
level of project implementation. It expands on the concept of project implementation contained in the 
previous report by including not only what equipment is in place at the schools but what level of usage 
that equipment is receiving. This section reviews teacher survey results, progress on telephone 
installation, and other items relating to project implementation. The next section evaluates how the project 
is succeeding in meeting its specified goals. The report concludes by presenting recommendations for 
reaching the project goals more effectively. 

Information for the report was gathered from several sources, including classroom observation, principal 
interviews, teacher interviews. District computer files, employee surveys, and campus computer logs. 
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STUDENT ACHIEVEMENT 



The 1991-92 school year marked the first full year of computer usage by the students. In line with the 
District's strategic objectives, the main purpose of tlie technology program is to improve student 
achievement. This section details changes in student achievement from the year prior to project 
implementation through the first two years of the project. The section presents information on student 
achievement from several angles: ROPE/ROSE, TAAS/TEAMS, ITBS/NAPT, and summer school. 



Defining ROPE and ROSE 

The four schools differ on many factors, and to compare achievement scores directly would be 
misleading. The Report on Program Effectiveness (ROPE) and the Report on School Effectiveness 
(ROSE) provide a more accurate interschool comparison of achievement results, ROPE and ROSE give 
information on how each school's students perform on standardized tests from one year to the next in 
relation to the rest of the District. Tlie reports do this by combining the individual scores of each student 
in a school or program. ROPE and ROSE adjust the scores for factors out of the school's control before 
making the comparison. These factors include sex, previous achievement, ethnicity, income level, and 
age in grade. 

ROPE and ROSE compare students' actual scores with a predicted score for each student. The difference, 
called a residual, is an indication of how far above or below prediction a student performed on a test 
compared to students with similar characteristics. The residuals of all students in a program are combined 
to create a program's ROPE score. The residuals of all students at a school are combined together to 
create a school's ROSE score. Three results are possible: exceeded predicted gain, achieved predicted 
gain, and below predicted gain. A score exceeding or below predicted gain is based on statistical tests 
to determine if the residual is significantly different from zero. 

This section presents the ROPE and ROSE scores for the four elementary technology demonstration 
schools. ROPE and ROSE generate scores only on students who took standardized tests the previous year. 
Therefore, kindergarten and grade 1 students are not included in the results. 
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Rope Comparisons 

To create the ROPE score, we generated a report for students who have recorded ROSE scores at the 
same four elementary schools for three consecutive years» from 1989-90 to 1991-92. Undt^r this 
condition, only groups of students in grades 4 and 5 in 1991-92 were large enough to meet the statistical 
requirement of having a minimum of 25 students to be included in this study. 

ROPE reported six scores for each program. Of the 24 possible changes, three exceeded the predicted 
gain and one was below the predicted gain. Figure 2 displays the scores for each school. 



Figure 2 
ROPE Scores 1991-92 






Andrews 


Gniindo 


Lnngford* 


Palton 


Grade 4 










Reading 










Malhcinalies 








t 


Language 










Grade 5 










Reading 






t 




Malhennalies 




t 






Lnngungc 











* In grade 4 nl Langford, only 22 sludenU qualified for maUicmalio.T and 23 for reading and language. 
Because ihis is below ihc ininitnum ot 25, ihcsc results should hz inlcrpretcd with more caulion. 

Key 

= Achieved Predicted Gain A Below Predicted Guin 
t Exceeded Predicted Gain n/a Test not Given 




The ROPE scores portray positively the effect of the computers on student achievement. ROPE, by 
looking only at students who have been influenced by the technology program for consecutive years, may 
be a better indicator of program effect than ROSH. Although 83.8% of the scores achieved the predicted 
gain, the ratio of three exceeding predicted gain to one below predicted gain is encouraging. 
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ROSE SCORKS AND COMPARISONS 



Fisure 3 displays ROSE scores for the year preceding project implementation and the two years since the 
beginning of the project. One way to look at the ROSE scores is to analyze how a school performed in 
relation to the predicted gain for that school. The technology at the four elementary schools is designed 
to help students exceed the achievement of students at regular schools. Correspondingly, we expect the 
ROSE scores for the four elementary schools to exceed the predicted gain when compared to scores at 
nontechnology schools. 



Figure 3 
ROSE Scores 
1989-90, 1990-91, and 1991-92 
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Galindo 


Langford 


Patton 
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90-91 


91-92 
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Reading 


























Malhemalies 




i 


; 










i 








t 


Language 






1 






1 




i 






i 




Grade 5 
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Malhemaliea 
Language 






t 
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t 
t 
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Nolc: The District switehcd to NAPT from FTBS in 1991-92 for grades 3-5. The NAPT does not measure 
work-study skills, so these seorcs from previous years hav e been dropped for this iniport. 

1= Achieved Predicted Gain i Below Predicted Gain 
It F.xcccdcd Predielcd Gain n/a Test not Given 



Each school received 10 scores-two each in grades 2 and 3 and three each in grades 4 and 5. The 
breakdown of the scores is provided in Figure 4. 

At Galindo, the Apple school, 20% of the scores exceeded the predicted gain and 10% were below the 
predicted gain. At the IBM schools, 10% of the scores exceeded the predicted gain while 16.7% of the 
scores were below the predicted gain. 
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Figure 4 
ROSE Scores 1991-92 





Andrews 


Galindo 


Langford 


Patton 


Exceed Predicted 


1 (10%) 


2 (20%) 


1 (10%) 


1 (10%) 


Below Prcdiclcd 


2 (20%) 


1 (10%) 


1 (10%) 


2 (20%) 


Equal Predicted 


7 (70%) 


7 (70%) 


8 (80%) 


7 (70%) 



Another way to look at the ROSE scores is to compare this year's scores with last year's scores. This 
comparison indicates movement between the three possible categories. For example, grade 4 mathematics 
at Langford was below the predicted gain in 1990-91 and equalled the predicted gain in 1991-92, This 
group improved its score but did not exceed the predicted gain, A comparison of the two scores credits 
this improvement, and shows progress toward achieving the level of exceeding predicted gain. 

Out of 10 comparisons at the Apple school (Galindo), from 1990-91 and 1991-92: 

• 2 up (20.0%) 

• 1 down (10.0%) 

• 7 same (70.0%) 

Out of 30 comparisons at the IBM schools (Andrews, Langford, and Patton), from 1990-91 and 
1991-92: 

• 7 up (23.3%) 

• 5 down (16.7%) 

• 18 same (60.0%) 

No changes occurred in the majority of the comparisons: 70% of the Apple school comparisons remained 
constant and 60% of the IBM school comparisons remained constant. The IBM schools registered a higher 
percentage of change (40%) than the Apple school (30%), but almost half of the IBM changes were 
negative. 

There were more changes this year than last year at both groups of schools and the percentage of positive 
change was higher than the percentage of negative change (see Figures 5 and 6), At the Apple school, 
the number of negative changes remained the same while positive changes doubled to two. At the IBM 
school, the number of positive changes remained the same while the number of negative changes 
increased from three to five. In other words, in 1991-92 there was an increase in the negative change 
compared with the first year of the program at the IBM schools and there was an increase in the 
positive change compared with the first year of the program at the Apple school. 

These two methods of analyzing the ROSE scores provide similar results for the Apple school. Method 
1 shows that 20% of the scores exceeded the predicted gain while 10% were below the predicted gain. 
Method 2 shows that 20% of the scores rose in comparison to 1990-91 while 10% decreased. 

The methods give different results for the IBM schools. The first method indicates that 10% of the scores 
exceeded the predicted gain while 16,7% were below the predicted gain. The second method shows that 
23.3% of the comparisons increased from 1990-91 to 1991-92, and 16.7% of the comparisons decreased. 
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Figure 5 

Changes in ROSE Scores, Apple School 
1991-92 




1989-90 to 1990-91 1990-91 to 1991-92 



Figure 6 

Changes in ROSE Scores, IBM Schools 
1991-92 




1989-90 to 1990-91 1990-91 to 1991-92 
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Figure 7 looks at the ROSE comparisons by grade and computer model, and shows at what grade levels 
and in what subjects the changes occurred. Figure 8 displays the grade in which the changes occurred 
for the three IBM schools. 



Figure 7 

ROSE Comparisons By Grade and Computer Model 
1990-91 and 1991-92 



Grade 


Apple School 


IBNi Schools 


2 


2 same 


i down {reading) 
5 same 


3 


2 same 


1 up (1 reading) 
3 down {} reading, 

2 niatheinatics) 

2 same 


4 


1 down {language) 

2 same 


4 up (2 mathetnatics, 

2 language) 

5 same 


5 


2 up {reading, 

inaihemancs) 

1 same 


2 up (2 reading) 
1 down {} reading) 
6 same 



Grade 5 registered both positive changes at the Apple school, one in reading and one in mathematics. At 
the IBM schools the changes are spread out among all grades, with the most changes (four) occurring in 
3rd (one up and three down) and 4th (four up) grades. Langford and Patton registered the most positive 
changes (three). 



Figure 8 
ROSE Comparisons By School 
IBM Schools, 1990-91 and 19P1-92 



Andrews 


Langford 


PaUon 


1 up {J 5th grade) 

2 down (2 Srd grade) 
7 same 


3 up (2 4ih grade, 
1 5th grade) 
1 down {1 3rd grade) 
6 same 


3 up {] 3rd grade, 
2 4th grade) 

2 down {1 2nd grade, 
1 5th grade) 

5 same 



Grade 4, where the highest number of positive changes occurred, consistently ranked at the top of all the 
tested grades at Patton and Langford in logged computer time (see p. 31). This points to a potential 
positive relationship between time on computer and improvement in test scores. However, grade 3 at 
Andrews registered two negative changes, and this grade is likewise consistently at or near the top in time 
on computer for this school. The evidence is inconclusive, but the potential link between logged computer 
time and achievement improvement exists. 
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To test the hypothesis that an increase in logged computer time leads to achievement improvement, we 
ran a regression analysis with ROSE residuals as the dependent variable and logged computer time in a 
particular subject as the independent variable. We ran this model for each subject and grade at each 
school. The model should tell if any statistical relationship exists between the two variables. This 
relationship is expressed in terms of R^ which is defined as the amount of variance explained by the 
independent variable. The expectation is that a positive relationship exists; that the higher a student's time 
on the computer in a given subject, the better the student scores in relation to the predicted gain. 

The analysis showed no demonstrable relationship between the two variables. The highest value 
obtained was .114 in grade 2 mathematics at Langford, which means only 11% of mathematics 
achievement is explained by time on compute; in mathematics for grade 2 at Langford. The remaining 
R2 ranged from 0 to .11. The failure of this model to confirm our expectations does not prove the 
expectations are invalid. Other factors, such as the uncertainty surrounding the logged computer times, 
may have negatively influenced the model. 

In sum, the ROSE and ROPE comparisons do not present convincing evidence regarding the effect of 
technology at the four campuses. The results are mixed at best. The ROPE scores portray the effect of 
computers more positively. Overally there is little proof as yet of the effectiveness of technology at the 
four campuses in improving student achievement. 
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Standardized Test Results 

During the 1991-92 school year, schools administered two types of standardized tests to students: a 
criterion-referenced test (CRT) to grades 3 and 5 and a norm-referenced test (NRT) to grades 1 to 5. As 
mentioned earlier, direct comparisons of these scores between schools can be misleading. 

TAAS/TEAMS 

The Texas Assessment of Academic Skills (TAAS), a CRT, was first administered In 1990-91. TAAS 
was preceded by the Texas Educational Assessment of Minimum Skills (TEAMS). Both are CRTs, which 
are designed to measure a welNdefined set of skills and to reference students' scores to a mastery criterion 
for that set of skills. The skills measured are a subset of the Essential Elements adopted by the State 
Board of Education. 

The switch to TAAS from TEAMS caused the percent passing districtwide to decrease because TAAS 
is more difficult. Additionally, the standards for passing were raised before the 1991 testing period, 
adding additional downward pressure on the percent passing. Thus, year-to-year comparisons should be 
made very carefully. 

Figures 9-11 present CRT percent mastery scores for the last three school years. Again, because of 
different sudent factors beyond the control of the school, these scores should De compared with caution. 
Additionally, with the switch from TEAMS, a minimum skills test, to TAAS, an academic skills test, 
change between years is difficult to interpret. 

An interesting trend appears if each school's scores are analyzed in relation to the District average. 

► Andrews exceeded the District average in two sections in spring 1990, no sections in fall 1990, 
and no sections in 1991. 

► Galindo exceeded the District average in no sections in spring 1990, three sections in fall 1990, 
and ^'our sections in 1991 (equalled in one section). 

► Langford exceeded the District average in five sections in spring 1990, one section in fall 1990 
(and equalled the average in two sections), and no sections in 1991 (equalled in one section). 

► Patton exceeded the District average in every section each year. 

The position of the IBM schools in relation to the District average worsened over the three-year period. 
The position of the Apple school improved markedly over the three-year period. 
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Figure 9 
Spring 1990 TEAMS Results 
Percent Mastery 
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Figure 10 
Fall 1990 TAAS Results 
Percetit Mastery 



Pwo«c Mu<£ry Grade 3 TAAS Results 

100-1 




' ■ ■ ir rorv^ , - ' ' ' ■ I- — 

Writing Reading Mathematics 



Percetu Uttuocj Grade 5 TAAS Results 




Writing Reading Mathematics 





Andrews 


Galindo 


Lang ford 


Patten 


AlSD 


GRADE 3 


Writing 


58 


71 




61 


80 


68 


Reading 


70 


82 




80 


95 


85 


Malhemalics 


73 


88 




72 


94 


87 


GRADE 5 


Writing 


54 


89 




82 


94 


82 


Reading 


45 


53 




76 


87 


69 


Malheinaties 


31 


52 




61 


83 


61 



12 




91.30 



Project A+ Elementary Technology Schools, 1991-92 



Figure 11 
Fall 1991 TAAS Results 
Percent Mastery 
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mS/NAPT 

Two NRTs were administered in Austin during the 1991-92 school year. The Iowa Tests of Basic Skills 
(ITBS) was given to grades 1-2 and the Norm-referenced Assessment Program for Texas (NAPT) was 
given to grades 3-11. An NRT is designed to measure student achievement in broadly defined skill areas 
that cover a wide range of achievement. Scores from NRTs (e.g., percentiles and grade equivalents) 
compare a student's performance with that of a nationwide sample c students at the same grade. National 
norms provided by the test publisher are used. 

ITBS /NAPT Scores by School, Grade, and Ethnicity 

To examine change in test scores across time we compared the ITBS/NAPT scores by ethnicity and grade 
from 1989-90 to 1991-92. This comparison will help determine whether the technology has had a 
differential impact across ethnic groups. The analysis of the test scores gives a measure of the 
performance of a school, not a specific group of students, by comparing scores from the same grade in 
successive years. The tables are presented in Appendix B on pages 34-35. 

Discerning a clear pattern from these numbers for a specific ethnic group and subject is difficult. Selected 
trends for specific grades and ethnic groups are presented belov;. 

• Andrews: Grade 1 all ethnic groups improved scores except Hispanics in language; African-American 

scores nearly doubled in reading and mathematics. 
Grade 3 African-American scores are down in reading and language; Hispanic scores are 

up in reading and mathematics. 
Grade 5 Hispanic scores are down in all subjects. 

• Galindo: Grade 3 mathematics scores are up for all groups; language scores for Hispanics are down. 

Grade 5 scores for Hispanics are all up. 

Composite scores for all ethnicities combined Increased in every grade from 1989-90 to 
1991-92. 

• Langford: Grade 1 composite scores are down for all groups, due to declining mathematics and 

reading scores. 

Grade 2 mathematics scores are down for all groups except African-Americans. 
Grade 4 African-American scores are down in all subjects. 

• Patton: In Grades 1 and 2 the majority of scores are down slightly across the board. 

Grade 3 composite scores are up, mostly on the strength of higher mathematics scores. 
Grade 4 composite scores are up. 

Analyzing the scores this way points to some positive gains made by some ethnic groups in some subjects 
at some schools. Looking at the composite scores from 1989-90 to 1991-92 it appears scores for all 
ethnicities combined are up at Galindo, down at Langford, and mixed at Andrews and Patton. African- 
Americans may have benefitted more than other ethnic groups thus far in the project although Hispanic 
scores have shown postive gains. Still, it is not possible to draw hard conclusions from the available data. 
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Summer School Program 
This section of the report considers two aspects of the summer school program: 

• What type of students were selected to participate in the summer school program? 

• Did the summer school students* test scores improve the subsequent year? 

Student Selection 

One of the strategies at the schools to keep students functioning successfully at or beyond grade level is 
to offer summer school classes to students not on grade level. Thus, student selection for summer school 
is a major concern. 

Ideally, 100% of the summer school students would be classified at risk. Figure 12 displays the percent 
of summer school students classified at risk by campus. The figure displays at-risk percentages for 1990- 
91 (the year prior to summer school) and for 1991-92 (the year after). The at-risk statistics are generated 
in October of each year, so by the time summer school student selection decisions are made in the spring, 
a student's actual at-risk status may have changed. Thus, the figure displays pre- and postsummer school 
at-risk statistics in an effort to portray more accurately the status of the students. 

In no school were 100% of the summer school students classified at risk. The postsummer school at-risk 
statistics were higher than the presummer school statistics, reflecting the change in at-risk status during 
the year. The percent of summer school students classified at risk ranged from a low of 66,1% at 
Andrews to 93.0% at Langford. 

Another way to evaluate summer school student selection is to compare the percent of summer school 
students classified at risk with the percent of school year students classified at risk. Reflecting the reason 
for summer school, to offer classes to students not on grade level, the summer school at-risk percentage 
should be much higher than the school year percentage. 



Figure 12 
At Risk Percent Comparison 
199! Summer School Students and 1990-92 School Year Students 
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The percent of summer school students classified at risk is higher at all four schools than the percent of 
school year students classified at risk. At Langford and Galindo the summer percent is more than twice 
as high as the 1991-92 school year percent, and at Patton the summer percent is over three times as high. 
Therefore, while 100% of th': summer school students were not at risk at any school, the percent at risk 
in summer school was higher than the percent at risk in the school year at all schools. 

Test Scores of Summer School Students 

The best way to analyze summer school students' test scores is to use the ROPE methodology described 
on pages 3-4. However, ROPE requires a minimum of 25 students for the results to acquire sufficient 
statistical confidence. The summer school student groups studied at the A-f schools ranged from a low 
of 9 to a high of 20. Nonetheless, the trends in the summer school ROPE data are so clear and consistent 
that they warrant reviewing (see Figure 13). 



Figure 13 
ROPE Scores 
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No group of summer school students in any grade at any school exceeded the predicted gain. Out of 
38 scores, seven (18.4%) were below the predicted gain. At the three IBM campuses, six (21.4%) of the 
scores were below the predicted gain. 
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Remember, these scores are not as statistically certain as would be preferred. However, the predicted 
gains are based on analyses of students with similar characteristics districtwide who had no concentrated 
technology program and did not have the benefit of summer school. The trend toward no groups 
exceeding the predicted gain and seven scores below the predicted gain may indicate a need to reconsider 
the purpose or activities of the summer school program. 

PROJECT IMPLEMENTATION 



This section of the report examines the level of project implementation reached as of the 1991-92 schoo' 
year. Last year's evaluation report found that the delayed arrival of the computers at the campuses 
Inhibited the evaluation of project success in terms of student achievement. This report has shown that 
students are not outperforming other District students at this point of the program. This section of the 
report asks the following two questions. 



Is it possible that the project is not yet fully implemented? 
If so, what parts of tlie project lack complete implementation? 



In a complex project of this magnitude, having all of the computers installed, networked, and loaded with 
software does not ensure implementation. Other important factors must converge to reach full 
implementation. Students must use the instructional software, teachers must integrate the software with 
the curriculum, teachers must receive sufficient and appropriate training. This section gives some 
indication of the level of implementation of these factors. 

LOGS OF Computer Use 

The IBM-networked equipment features a logging system to track computer usage by students. This 
information can be partitioned into various categories, including grade and subject area, Prior to the 
summer session 1991, the log information was not usable by District personnel because students logged 
in with their name instead of the identification number. Following a switch to logging on wnh the 
identification number, summer session 1991 marked the first time we collected the log information. We 
also collected the logs during the 1991-92 school year. 

The log system operates as follows. Individual teachers create student menus for their classroom by 
selecting courseware and inserting it in the menu under the appropriate subject, such as mathematics or 
reading. The teacher is given the option of recording logs for each courseware selected. The teachers 
have been instructed to chose the no-log option for some courseware, such as Writing To Read and 
various third-party courseware for which the system is not capable of keeping logs. 

When initiating a computer session, students log on by entering their AISD identification number. The 
logging system then automatically records several pieces of information, including the time in. the time 
out, and the name of the courseware the student uses. 
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The evaluation associate (EA) collected the logs each six weeks by downloading them to computer 
diskettes. A programmer uploaded the diskettes to the AISD mainframe and analyzed the data. The EA 
sent preliminary analyses three times during the year-one covering the first and second six weeks, one 
covering the third and fourth six weeks, and one covering the fifth and sixth six weeks. For further detail 
on the log files and to see the preliminary analyses, see Project /14- Elementan Technology 
Demonstration Schools Technical Report (ORE Publication No. 91 .N) 

Several problems were encountered with the logging system. These problems can be grouped into two 
areas: user problems and technical problems. User problems include teachers inappropriately selecting 
the no-log option, students being deleted from the log files before the logs are downloaded, and students 
not logging in with their District identification number. Technical problems include system upgrades that 
erased logs, the logging system not tracking times appropriately, downtime on the servers causing log 
loss, and the system's inability to log multiple identification numbers for students working in pairs. 

These problems added uncertainty to the information provided by the logging system. Many of the 
campus personnel and teachers expressed strong disagreement with the times the logs showed. 

The technology facilitator called a meeting to discuss the preliminary analysis of the log information from 
the first and second six weeks. This meeting generated some positive results. The campus lab assistants 
began maintaining records of fileserver downtime to document technical problems better. IBM personnel 
ran simulations to see if the system was logging times correctly. 

Subsequent atwlyses of logged computer time showed little improvement in time or? task from the first 
analysis. Campus personnel began rethinking their perceptions of time the students spent on the 
computers, and some adjusted their perceptions downwards. Others showed continued resistance to the 
time-on-task number derived from the logs. 

Despite the controversy over the logs, there are several reasons that compel their inclusion in this report. 
Available evidence does not yet single out the factor or factors causing a malfunction. Because there is 
as yet no determined pattern of logging system malfunction, we can assume that the logs are useful for 
interschool and intraschool comparison purposes. Finally, the logs are central pieces of information for 
evaluating the technology program. 

Students do spend some of their time on software that is not logged. The grades most affected by this are 
the grades that use Writing to Read, primarily kindergarten and grade I, and Writing to Write, primarily 
grade 2. Additionally, it is not certain if the logging system maintains accurate records for some of the 
third-party packages for which logs are kept, namely the DLM software (Number Farm, Comparison 
Kitchen, and Alphabet Circus). Nonetheless, only a small minority of overall students' time is spent on 
these packages. Figure 14 lists the logged software and gives explanations for definitions in the tables and 
graphs that follow. 
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LogRed Courseware at IBM Schools and 
TkflnUlomfor Log Tables 



Malhcmaiics 



Language 



Malh Concepts 
Math Practice 
Geometry 
Algch ra 

Measurement, Time, and Money 
Comparison Kitchen 
Number F-nrm 



Spelling 
Vocabulary 
Parts of Speech 
Punctuation 
Combining Sentences 
Alphabet Circus 
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Bouncy Bee Learns Letters 
Bouncy Bee Learns Words 
Stories and More 
Reading for Meaning 
Reading for Information 



Primary Editor Plus 
Mi Editor Primario 
Typing 

TouchTypingfor Beginners 



Tools 



The Writing and Publishing 
Center 
LANSchooI 
Link Way 



DKI'INITIONS 



All Grades Colnnui: 

This column is the mean lime for ail students with records in » specific subject. The count is unduplicuied. 



All Sn!)jec(s Row: 

This row is the mean time for all students with records in a specific grade. The count is unduplicated. 
All Subjects and All Grades Cell: 

This cell is the mean for all students with records in any subject. The count is unduplicated. This can be loosely 
interpreted as the grand total of time spent on the computer per Situdent for that time period. 

First and Second Six Weeks Combined: 

For individual subjects: Includes those students with records for that subject in both six weeks. 
For all subjects: Includes students with records in any subject in both six weeks. In some cases, this causes the all 
subjects total to exceed the sum of the individual subjects for a specific grade in the first and second six weeks 
combined table. The counts are unduplicated. 



Because of the uncartainity associated with the logs, the absolute times on task shown in Figure 15 and 
in Appendixes A1-A3 should be interpreted with caution. The times are far below expectations for a 
program of this magnitude. E>'en if the logged times were doubled, the amounts would still be below 
expectations. The highest average daily time of any six weeks period was 19.0 minutes. The reported 
logged times may serve to stimulate time-on-task goal setting where such goals do not exist, and lo 
prompt further study about time goals where they do exist. 




91.30 



Project A+ Elementary Technology Schools, 1991-92 



The amount of logged time peaked at the campuses during the second and third six weeks and levelled 
off or declined thereafter. The peak logged time varied from campus to campus, but as mentioned above 
the maximum was 19 minutes per day. This low demonstrated level of use raises questions about the level 
of project implementation. In interviews, principals reported that some campuses do not have time targets 
for computer use. These targets are necessary. Moreover, reaching these targets must be enforced and 
monitored if the effectiveness of technology in reaching the project goals is to be tested. The computer 
logs provide excellent information to monitor use for this purpose, and they must be made to work 
correctly. 



Figure 15 

Average Minutes Per Day on Computer at All 
IBM Elementary Technology Demonstration Schools 
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Two models of computer use are presented in Figure 16. Because the school day lasts a total of six and 
one-half hours, excluding lunch, the maximum amount of time that can be achieved under these 
assumptions is 1 hour and 20 minutes per student. To achieve this, the computers must be in continuou 
use the entire day. Based on the information contained in the logs and obtained by observation, it is 
doubtful any cl assrooms are reaching this level of use. 

Figure 16 

Models of Daily Student Computer Use 

These models illustrate typical scenarios of classroonn computer usage. The models 
compute how much time computers must be in use in order to reach certain time 
goals. 

ASSUMPTIONS 
22 Students 
4 Classroom Computers 
45 minutes lab use per student per week (990 minutes total) 
6.5 hours per school day 

MODEL 1: Target of 1 hour per student per school day 

22 students x 1 hour each = 22 hours total daily usage 

22 hours x 5 days = 110 hours (6,600 minutes) pci week 

6,600 minutes - 990 lab minutes = 5,610 minutes (93.5 hours) classroom usage 

93.5 hours usage 5 days - 18.7 hours per day 

18.7 hours per day 4 classroom computers = 4.7 hours per computer 



CONCLUSION: For 22 students to use the computers one hour each, 
all four classroom computers would have to be in continuous usage 4 
hours and 42 minutes every day, or about three-fourths of the school 
day, 

MODEL 2: Target of 30 minutes per student in mathematics, reading, and 
language arts (total of 90 minutes per day) 

22 students x 1.5 hours each = 33 hours total daily usage 

33 hours x 5 days = 165 hours (9,900 minutes) per week 

9,900 minutes - 990 lab minutes = 8,910 minutes (148.5 hours) classroom usage 

148.5 hours usage 4- 5 days = 29.7 hours per day 

29.7 hours per day 4 classroom computers = 7.4 hours per computer 



CONCLUSION: It is not possible for 22 students to use the computer 90 
minutes daily without spending more than 45 minutes weekly in the 
laboratory or without ha ving more than four classroom computers, 

WUh these assumptions, the maximum amount of time each student could spend 
on a completer is 1 hour 20 minutes per day if the computers were continuously 
used. 
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Employee Survey Responses 

Every fall the District conducts a survey of all employees. Fall 1991 marked the second time Project AH- 
staff has responded to the same set of questions. Responses from both years are displayed in Figure 17. 

Several observations are worth noting. The percent of neutral responses decreased in every item except 
item 2. In most cases this decrease appeared in the agree and strongly agree responses. The major 
exception is item 3, which asks if the training was sufficient to appropriately incorporate the Project A + 
technology into the curriculum. The percent of persons with negative responses to this item rose from 
14.6% to 19.7%, the highest negative response. One in five people think the training is not sufficient 
to incorporate the A+ technology appropriately into the curriculum. 

Only item 2 about retention rates experienced a drop in the percent responding positively. This decrease 
may be attributable to the increased emphasis by the District on reducing retention rates, which 
overshadows the effect of the technology project on reducing retention rates. 

Respondents were very encouraged by the academic progress made by the students using the technology 
(item 1). This question netted the highest positive response rate, although this supposition is not yet borne 
out by the data. Respondents also believed the addition of technology in the classroom had made their 
teaching more effective. 

Campus Computer Issues 

Teachers at the IBM schools completed training in the software delivery system. Teaching and Learning 
with Computers (TLC), during fall 1991. Teachers implement this delivery system with varying degrees 
of success and enthusiasm. Some teachers use the system almost exclusively in mathematics and language 
arts; some find the delivery system too cumbersome and use it very little. 

The purpose for which the computers are used in the classroom is a major issue. Sometimes computer 
time is still viewed as a reward or the computer is used as a discipline tool. In interviews, one teacher 
stated that students perform their regular work as quickly as possible so they can get on the computer. 
Another stated that students that behave well can earn computer time. 77?/^ type of usag'B should be 
eliminated. The computers are for all students to use. Rewarding well-behaved children or quick workers 
works against some of the project goals of accelerating the learning of students not on grade level and 
may actually even widen the gap between at-risk and not-at-risk students. 

Another important issue in intergrating computers into the curriculum is the teaching method used in the 
classroom. One teacher commented, "There has been no effort to integrate computers in a structured 
way." Several teachers report still relying exclusively on direct teaching. One teacher tells of students 
hand writing stories on paper, then typing them in the computer. 

The project is not designed to eliminate direct teach. Nor is the aim to tell teachers how to integrate the 
computers mto the curriculum. The project is designed to integrate the computers into the curriculum in 
a structured way to improve student achievement. The examples listed above are inappropriate uses of 
the technology. This inappropriate use may be caused by several factors, including insufficient training 
or lax supervision. Computer usage needs to be structured, monitored, and consistent for its effect to 
be measured. 
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Figure 17 
1990-91 and 1991-92 Employee Survey 
Teacher Responses 
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Strongly 
Agree 
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Disagree 


Strongly 
Disagree 


1991 1992 


1991 1992 


199! 1992 


1991 1992 


1991 1992 


1991 1992 


1. I am encouraged by the academic 
progress the students on my campus have 
made using the Project A+ Technology. 


189 176 


30.1 36.5 


28.4 40.1 


34.4 17.4 


4.4 4.2 


2.7 1.8 


2. The implementation of the Project A + 
Technology will help reduce retention 
rates on my campus. 


190 179 


26.2 20.6 


41.5 40.0 


26.2 32.9 


5.5 5.3 


0.5 1.2 


3. I received sufficient training to 
incorporate appropriately the Project A + 
technology into my curriculum. 


193 176 


21.0 27.4 


40.3 38.7 


24.2 14.3 


8.1 13.7 


6.5 6.0 


4. The addition of technology in my 
classroom has made my teaching more 
effective. 


193 174 


23.1 31.1 


32.8 37.7 


34.4 20.4 


7.5 7.8 


2.2 3.0 


5. I would recommend the Pioject A + 
technology as it was implemented on my 
campus. 


193 175 


25.3 33.5 


40.9 37.1 


22.0 18.0 


9.1 8.4 


2.7 3.0 

- 



I) 

2) 
3) 



OBSERVATIONS 

Teachers expressed the most agreement with item 1 and the least agreement with item 2. 
Teachers expressed the least disagreement with item 1 and the most disagreement with item 3. 

The number of respondents answering neutral decreased, as teachers lend to feel more strongly about the project. This is 
seen in the increase in the percent of respondents agreeing or disagreeing for every item but number three. For item 1 the 
number disagreeing decreased. For item 2 the number agreeing decreased. For item 5 the number disagreeing decreased. 
The number of neutral responses rose only for item 2. 
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Some teachers also expressed concerns about the type and quality of the courseware being used. The 
computers are capable of much more than drill-and-practice, but higher order activities require 
appropriate courseware. A teacher at one of the IBM schools claimed that Typing Tutor was the only 
non-drill-and-practice courseware available to the students. For the computers to reach full effectiveness, 
courseware that utilizes higher order skills is required. 

Technical problems with the computers continued. The maintenance contract with IBM expired during 
the year. Maintenance switched to the District, but service delivery problems engendered a brief return 
to IBM-provided maintenance. At the end of the year, the District once again picked up maintenance on 
the computers. Principals report improved maintenance service delivery at this point. 

IBM is converting all computers to token ring. Some of the initial wiring was baseband, which was too 
slow and caused technical problems. The switch is anticipated to be complete by the beginning of the 
1992-93 school year. 

Training 

Training for teachers at all schools continued during the 1991-92 school year, but most of it was follow- 
up. The major new training was for teachers at the IBM schools in Teaching and Learning with 
Computers for mathematics from October 29 to November 5, 1991, at IBM. Other training included 
Writing to Read October 22 and 23 at Langford, Writing to Write November 7 and 8 at Langford, File 
Management November 7 at Galindo, and HyperCard February at Galindo. In addition, IBM offered a 
staff development session October 14 at the Red Lion Hotel. 

Approximately two teachers from each campus were trained in the Excelsior II electronic gradebook 
program. These teachers then served as campus resources, fielding questions from other staff members 
on the program. 

The amount of follow-up training may need increasing. On the annual staff survey, teachers expressed 
the most disagreement with the statement that they received sufficient training to incorporate appropriately 
the Project A+ technology into their curriculum. In personal conversations, many teachers voiced their 
opinion that training now would be more useful than the training they received at the beginning. 
Increasing follow-up training would aid overall project implementation and improve project 
effectiveness. 

Parent Take-home Computers 

The three IBM technology schools began implementing the parent take-home computer program during 
the school year. The program allows selected students to take home computers loaded with specific 
instructional software designed to accelerate the students' learning. Each campus implements this program 
slightly differently. The ETDS technical report contains the campus take-home computer plans. 

At this point there is no evaluation of the effectiveness of the take-home computer program. The schools 
monitor usage of the computers. 
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Telephone Installation 

The original project plan proposed to place telephones in every classroom to facilitate parent-teacher 
communication. Although the telephones are not yet installed, some progress toward their installation was 
made this year. 

The telephones will operate on the same system as the central administration telephones. Galindo piloted 
the phone project during the year. Several staff members received telephone training and approximately 
five telephones were installed. At a February 1992 meeting the School Board approved a bid of $270,274 
for the equipment. At a June meeting the Board approved a lease-purchase agreement. 

As of this writing, telephone installation is just beginning. The system is scheduled to be fully operational 
by the opening of the 1992-93 school year. 

Restructuring Issues 

This report, like the project to date, emphasizes the role of technology and the computers. The computers 
are important to the success of the project, but they are properly viewed as only one part of an overall 
effort to enhance the learning of all students so that they are functioning at or beyond age appropriate 
grade level. An integral part of the overall effort is restructuring of the classroom and school. The aim 
of this restructuring its to enhance the learning of all students and to ensure that all students are 
functioning successfully at or beyond grade level. 

During the first year of the project, 1990-91, project staff conducted regular meetings to discuss issues 
related to the initial implementation of technology. These meetings were abandoned during the 1991-92 
school year. The results contained in this report indicate similar meetings could be useful. The meetings 
need to have the overall project goals as their focus. Specifically, participants should ask what can be 
done to better support student success at their schools and how to better meet project goals. Appropriate 
solutions may or may not deal with technology; for example, developing better mechanisms to accelerate 
an overage student's progress may include programs to advance the student through the grade levels more 
quickly. 
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PROGRESS TOWARD PROJECT GOALS 



Page 1 of this report listed the four specific goals of ETDS. Now at the end of the second year of the 
project, it is an appropriate time to evaluate progress toward the fiilfillment of these goals. This section 
analyzes progress toward these goals one by one. 



Goal 1: In three yean, reduce by 50% the number of students who are not in their age- 
appropriate grade level. 



The percent of overage students is decreasing. The number of overage students has decreased only at 
Galindo. 

Appendix CI contains the number and percent of overage students as of October 30 for each of the four 
schools. Only one school, Galindo, shows a decrease (from 106 to 90) in the total number of overage 
students from 1990 to 1991. The percentage overage decreased slightly, ranging from -.07 to -2.8, at 
three schools. This measure counts all students in the school on October 30 each year, without 
considering if they have been in the school less than the entire time of the program. This dilutes the 
measure of program effect by looking at students who have not had time to be affected by the program. 

Page 4 describes a ROPE file that contains students who have attended the same school for three 
consecutive years which corresponds with the year before the project and the first two years of the 
project. We extracted yearly overage statistics on this group of students to test the completion of the first 
goal (see Figure I8j. 



Figure 18 
Overage Statistics 
For Three-Year Students 







1990 


1991 


1992* 






N 


% 


N 


% 


N 


% 


Andrews 


66 


9 


13.6 


9 


13.6 


9 


13.6 


Galindo 


70 


9 


12.9 


9 


12.9 


9 


12.9 


Langford 


50 


6 


12.0 


6 


12.0 


6 


12.0 


Patten 


210 


22 


10.5 


22 


10.5 


22 


10.5 



* Preliminary data 



Of students at the schools three consecutive years, the number overage has remained constant. Of all 
students at the school, the number overage has remained fairly constant but the percent overage has 
decreased slightly due to increased enrollment (see page 36). Existing mechanisms to accelerate an 
overage student's movement through the grades appear insufficient to reach fulfillment of this goal. 
However, the number of new retainees has decreased. The reduction in the number of new retainees may 
be a result of the districtwide emphasis on reducing retainees and not the project. 
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Goal 2: In three years, reduce by 50% the number of students who are not achieving on grade 
level in reading, writing, and mathematics. 



The schools are not on pace to achieve Goal 2. 

The numbers and percentages of at-risk students at all schools increased from 1990-91 to 1991-92 (see 
Appendix C2). To get a more accurate picture of program effect, we analyzed the same ROPE file of 
students at the schools three consecutive years to test progress toward Goal 2 (see Figure 19), 



Figure 19 
Percent Below 30th Percentile in 
Reading and Mathematics^ Three-Year Students 



READING 


1990 


199 1 


1992 


N % 


N % 


N % 


Andrews 


66 


8 12.1 
11 15.7 

8 16.0 
22 10.5 


21 42.0 

15 21.4 
17 34.0 

16 7.6 


24 36.4 
10 14.3 
12 24.0 
8 3.8 


Galindo 


70 


Lang ford 


50 


Patton 


210 




MATHEMATICS 


1990 


1991 


1992 


N % 


N % 


N % 


Andrews 


66 


15 22.7 
8 11.4 
6 12.0 
5 2.4 


24 36.4 
8 1 1.4 
14 28.0 
10 4.8 


28 42.4 
12 17.1 
12 24.0 
II 5.2 


Galindo 


70 


Lang ford 


50 


Patton 


210 



The numbers in Figure 19 are derived from scores on the ITBS/NAPT. Based on these scores, only 
Patton is on pace to achieve Goal 2 in reading and no schools are on pace to achieve Goal 2 in 
mathematics. 
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is by looking at scores on the TAAS/TEAMS (see 



Figurt 20 

Percent Failing Reading, Maihematics, and Writing Section of 
TAAS/TEAMS, Three-Year Students 



READING 




1989-90 


1990-91 


1991-92 






N 


% 


N 


% 


N 


% 


Andrews 


66 


4 


6.1 


9 


13.6 


22 


33.3 


Galindo 


70 


9 


12.9 


7 


lO.O 


10 


14.3 


Lang ford 


50 


2 


4.0 


5 


10.0 


16 


32.0 


Pallon 


210 


4 


1.9 


5 


2.4 


18 


8.6 


MATHEMATICS 


1989-90 


1990-91 


1991-92 






N % 


N 


% 


N % 


Andrews 


66 


5 


7.6 


7 


10.6 


28 


42.4 


Galindo 


70 


5 


7.1 


5 


7.1 


9 


12.9 


Langford 


50 


1 


2.0 


5 


10.0 


18 


36.0 


Patton 


210 


2 


1.0 


5 


2.4 


25 


11.9 




wRnriNG 




1989-90 


1990-91 


1991-92 






N % 


N 


% 


N % 


Andrews 


66 


1 


10.6 


16 


24.2 


21 


31.8 


Galindo 


70 


10 


14.3 


16 


22.9 


16 


22.9 


Langford 


50 


1 


2.0 


7 


14.0 


13 


26.0 


Patton 


210 


7 


3.3 


21 


lO.O 


24 


11.4 



The number of students failing the reading, mathematics, or writing section of the TAAS/TEAMS has 
increased at all schools during the project. The percent failing has also increased at all schools. Galindo, 
the Apple school, has increased less than the IBM schools. 

It is important to remember that the TAAS, first administered during the 1990-91 school year, is a more 
difficult test than the TEAMS. Also, the passing standards were raised prior to the 1991-92 school year. 
Still, the increase in the number of students failing a section of the TAAS is high. The schools are not 
on pace to achieve Goal 2 of the project. 
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Goal 3: Develop a comprehensive teacher training program to ensure effective implementation 
and classroom use of technology. 



Information from the employee survey indicated that one in five teachers believed the amount of training 
was insufficient to incorporate the Project A+ technology into the curriculum. From personal 
conversations, many teachers believed the initial training was good but more follow-up training is needed. 
There is progress toward completion of this goal, but more follow-up training needs to be provided. 



Goal 4: Demonstrate to the community the educational benefits of technology, thereby obtaining 
support for d istrict wide implementation. 

As a demonstration project, the elementary technology demonstration schools aim to prove that 
technology is a viable method of instructing the children of Austin. The District is proceeding with plans 
to increase the amount of technology at all campuses. Funds for technology are forthcoming from the 
State. 

Many members of the community are excited about the technology programs at these four campuses. 
Principals report increased parental interest in their children's scholastic activities since the initiation of 
the program. Many students were turned away at the beginning of the school year who do not live within 
the schoors boundaries and have not procured the necessary transfers to attend the school. There has been 
some media coverage of activities at the school, all of it positive. 

At this point evidence for the educational benefits of the technology, as measured by performance on 
standardized tests, is minimal. The project does have considerable support within the community, but 
this support is derived more from intuition about the benefits of technology than from its demonstrated 
effects. 
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CONCLUSIONS AND RECOMMENDATIONS 



The 1991-92 school year was the second year of the elementary technology demonstration schools project 
and the first ftill year of computer usage by the students. The evidence is not yet clear on the project's 
effects, but some trends are beginning to emerge. 

Student achievement for students taking summer school classes is below expectations. Student 
achievement for students in the schools three consecutive yeai'S indicate some positive gains. However, 
analyses of the at-risk status of the same students show the number overage is holding steady, while the 
number below the 30th percentile in reading and mathematics is rising as is tiie number failing the 
reading, writing, or mathematics section of the TAAS/TEAMS. 

At this point, it appears that students at Galindo, the Apple school, have improved achievemeiU more 
than students at the IBM schools. This is based on a higher percentage of ROSE scores exceeding the 
predicted gain, better improvement in TAAS/TEAMS compared to the District average, better movement 
towards reducing the number of overage students, and a smaller increase in the percent failing a section 
of the TAAS/TEAMS. The difference in success in reaching the project goals may be attributable to the 
differences in instructional technology approaches and to the short time horizon of the program thus far. 
The mixed approach used at the IBM schools may require more time to fully implement, and after full 
implementation the approach may generate more positive results. At this point it is not possible to say 
which approach has a greater long-term effect. 

Based on the information contained in this report, we make the following recommendations. 

1. The computer technology needs to be fully implemented. Full implementation includes 
all students using the computers in an instructional setting on a regular, daily basis for an 
amount of time specified by the school. 

2. Schools need to set time-on-task goals and monitor their completion. Where such goals 
do not exist, principals need to lead the staff to setting the goals. 

3. Teachers need to use an instructional delivery system that incorporates the computers in 
instructional activities, 

4. The logging system needs to function accurately. All technical and user problems need 
to be resolved. Use of the computers needs to be monitored to ensure project 
implementation. 

5. Follow-up training for teachers needs to be increased. 
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APPENDIXES 



Appendix Ah Avi;r\gk Minutbs Per Day on Computer at Andrews Elementary 




Grade 


1st Six Weeks 


2nd Six Weeks 


3nJ Six Weeks 


4th Six Weeks 


5th Six Weeks 


6th Six Weeks 


K 


1.4 


4.8 


6.0 


6.8 


6.0 


4.7 


I 


4.5 


16.1 


19.0 


17.4 


17.2 


13.7 


2 


3,5 


10.6 


12.4 


14.5 


13.9 


12,6 


3 


7.0 


13.5 


16.8 


18.0 


14.7 


5,2 


4 


6.6 


8.5 


12.2 


9.9 


6.2 


6,0 


5 


4.8 


9.7 


12.8 


11.0 


8.7 


6.0 


All 


4.8 


11.2 


13.2 


12.4 


11.0 


8.6 



Note-.: The gmde 1 analysis excludes the students of one teacher for the first through fourth six weeks due to improper logging 
procedures. Memory and tape backup problems may have caused loss of some logs from server 1 during the third and fourth six 
weeks. Token ring and wiring problems caused some servers to be down at various times. 
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API'I'NDIX A2: AVl'RAGL MINUTES PlC R DAV ON COMPUTER AT LANGFORI) ELEMENTARY 




Grade 


1st Six Wcclcs 


2nd Six Weeks 


3rd Six Weeks 


4lh Six Weeks 


5lh Six Weeks 


6th Six Weeks 


K 


3.7 


4.1 


3.0 


2.5 


2.3 


1.5 


1 


3.7 


13.2 


13.0 


11.8 


11.0 


6.2 


2 


2.3 


7.7 


7.4 


9.7 


9.3 


5.2 


3 


6.2 


7.5 


6.6 


9.3 


8.9 


8.8 


4 


11.2 


16.1 


15.6 


17.0 


17.6 


7.7 


5 


10.5 


19.0 


15.8 


15.3 


12.8 


7.1 


All 


6.8 


11.4 


10.6 


11.8 


11.2 


6.7 



Notes: Problems with server 3 in the second six weeks may huvc eaused a loss of some logs. Servers 1 and 3 were down at times 
with net mail problems during the third and fourth six weeks. 
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APPENDIX A3: Average Minutes Per Day on Computer at Patton Elementary 



Patton Average Minutes per Day 



By Grade, All Subjects 




-T- 1 ■ 1 1 * 

1 2 3 4 5 6 

Six Weeks Period 



Gmdc 


Isl Six Weeks 


2nd Six Weeks 


3rd Six Weeks 


4lh Six Weeks 


5lh Six Weeks 


6lh Six Weeks 


K 


1.2 


6.0 


4.6 


3.5 


5.0 


2.9 


1 


7.0 


13.0 


13.6 


8.7 


16.8 


17.6 


2 


0.6 


9.3 


5.8 


6.0 


4.1 


4.5 


3 


0.8 


13.7 


13,6 


12.4 


13.9 


11.6 


4 


7.4 


18.8 


16.6 


13.0 


9.5 


13.6 


5 


1.4 


9.3 


8.8 


5.6 


5.4 


3.7 


All 


3.5 


12.0 


10.8 


8.3 


9.3 


9.3 



Notes: Server 4 crashed at the beginning of the year, causing loss of some logs. All servers were down several days in January 
and installalion of incorect drivers caused intermittenl problems during January. 
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Appendix B: NAPT/ITBS Scores 1989-90, 1990-91, and 1991-92 



Composite 



Mathematics 



Reading 



Language 



All H AA O 



Andrews 

Grade I 

1989- 90 

1990- 91 

1991- 92 



All H A.A 0 



All H AA O 



All H AA O 



Grade 2 

1989- 90 

1990- 91 

1991- 92 



51 68 



63 
32 
46 



Grade 3 

1989- 90 

1990- 91 

1991- 92 



54 53 



Grade 4 

1989- 90 

1990- 91 

1991- 92 



Grade 5 

1989- 90 

1990- 91 

1991- 92 



69 



78 



63 



68 



73 



6C 



Galindo 

Grade 1 

1989- 90 

1990- 91 

1991- 92 



Grade 2 

1989- rO 

1990- 91 

1991- 92 



57 



68 67 



Grade 3 

1989- 90 

1990- 91 

1991- 92 



Grade 4 

1989- 90 

1990- 91 

1991- 92 



49 49 



Grade 5 

1989- 90 

1990- 91 

1991- 92 



35 
46 



An underlined number means fewer than 10 students took that test. 
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Appendix B: NAPT/ITBS Scores 1989-90, 1990-91, and 1991-92 (cont.) 





Composite 


Mathematics 


Reading 




L 


angu 


age 






All 


H 


AA 


0 


All 


H 


^A 


0 


All 


H 


AA 


O 


All 


H . 


AA 


/-A 
U 


Langford 


































Grade 1 


55 


51 


54 


60 


50 


48 


43 


54 


44 


42 


42 


47 


j4 


50 


54 


59 


1989-90 


46 


36 


37 


56 


51 


41 


35 


63 


28 


19 


Zi 


37 


47 


40 


37 




1990-91 


42 


36 


32 


53 


29 


26 


16 


41 


29 


26 


18 


39 


56 


CO 

JZ 


CO 

jZ 


<^ 1 
O i 


1991-92 






























Grade 2 




























51 


30 


62 


1989.90 


53 


49 


23 


73 


eft 

59 


J4 




/o 


AT 




Zw 


61 


51 


1990-91 


48 


45 


35 


59 


51 


48 


36 


63 


43 


39 


34 


51 


43 


43 


34 


47 


1991-92 


44 


33 


38 


52 


42 


38 


28 


49 


35 


23 


37 


43 


48 


33 


51 


55 


Grade 3 




























63 


58 


69 


1989 90 


52 


41 


38 


65 


44 


34 


29 


59 


50 


36 


42 


64 


65 


1990-91 


55 


53 


31 


69 


49 


50 


22 


64 


51 


49 


26 


71 


66 


68 


52 


71 


1991-92 


4? 


47 


38 


55 


41 


42 


33 


43 


42 


44 


30 


45 


50 


52 


41 


51 


Grade 4 






























45 


72 


1989-90 


49 


42 


37 


70 


47 


40 


35 


72 


43 


37 


33 




55 


50 


1990-91 


36 


27 


24 


58 


34 


26 


25 


49 


37 


28 


26 


55 


41 


34 


27 


59 


1991-92 


45 


44 


16 


73 


44 


46 


19 


63 


41 


45 


14 


60 


50 


54 


37 


53 


Grade 5 




























52 


53 


/I 


1989-90 


54 


45 


45 


68 


54 


52 


38 


66 


49 


36 


44 


65 


60 


1990- 91 

1991- 92 


51 
50 


45 
40 


36 
39 


68 
69 


50 
43 


44 
36 


38 
33 


68 
59 


49 
47 


42 
41 


35 
38 


68 
61 


53 
60 


50 
52 


47 
jj 


61 
/ i 


Patton 


































Grade I 

1989- 90 

1990- 91 

1991- 92 


85 
82 
81 


73 
79 
6i 


77 

Zi 


86 
82 
82 


86 
80 
77 


73 
73 
£7 


Zi 

60 
78 


87 
81 
78 


79 
74 
73 


74 
72 
57 


75 
58 
62 


80 
75 

/4 


o< 
/J 

74 

1 1 


66 
75 

Oo 


73 
85 

70 

/z 


76 
74 
77 


Grade 2 






























45 


78 


1989-90 


84 


73 


47 


87 


89 


85 


70 


90 


77 


61 


40 


80 


75 


58 


1990- 91 

1991- 92 


84 
83 


71 
81 


§3 
78 


86 
83 


86 
82 


76 
75 


79 

Zi 


87 
83 


79 
77 


64 
75 


59 
62 


82 
77 


76 
77 


70 
86 


34 

2i 


78 
75 


Grade 3 




























75 
83 


79 


84 
87 


1989- 90 

1990- 91 


78 

9. 1 
o 1 


60 


64 
70 


82 


/J 

77 


jZ 

68 


yj 

66 


7*^ 
78 


73 
76 


56 
72 


55 
68 


75 
77 


83 
86 


1991-92 


89 


79 


70 


91 


83 


71 


64 


84 


79 


68 


56 


81 


84 


76 


64 


86 


Grade 4 




























56 
62 
83 


62 
67 
83 


76 
75 
84 


1989- 90 

1990- 91 

1991- 92 


74 
73 
88 


52 
48 
83 


57 
6i 
80 


77 
77 
89 


74 
74 
79 


49 
44 
76 


49 
63 
72 


78 
78 
80 


70 
70 
74 


51 
43 
67 


55 
59 
66 


74 
73 
76 


73 
74 
84 


Grade 5 


























78 


70 




79 


1989-90 


78 


61 




79 


78 


53 




80 


74 


56 




77 


63 


1990-91 


77 


52 


73 


81 


79 


51 


82 


82 


73 


47 


66 


77 


77 


59 


79 


1991-92 


82 


52 


83 


85 


71 


51 


59 


74 


67 


46 


59 


70 


78 


70 




79 



An underlined number means fewer than 10 students took that test. 
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Apfkndix CI: Numbers and Percent of Overage Students 



Andrews 







October 30, 1990 


October 30, 1991 




Grade 


Enrollment 


Overage % 


Overage 


Enrollment 


Overage % 


Overage 


K 


110 


5 


4.5 


136 


0 


0.0 


1 


113 


8 


7.1 


124 


9 


7.3 


2 


105 


15 


14.3 


126 


15 


11.9 


3 


108 


23 


21.3 


111 


24 


21.6 


4 


111 


34 


30.6 


111 


31 


27.9 


C 


92 


25 


27.2 


125 


37 


3S.6 


Total 


709 


110 


15.5 


833 


116 


13.9 


Galindo 




October 30, 1990 




October 30, 1991 




Grade 


Enrollment 


Overage % Overage 


Enrollment 


Overage % 


Overage 


K 


125 


1 


0.8 


120 


0 


0.0 


1 


128 


18 


14.1 


119 


5 


4.2 


2 


99 


16 


16.2 


122 


18 


14.8 


3 


113 


31 


27.4 


90 


22 


24.4 


4 


79 


20 


25.3 


93 


24 


25.8 


5 


85 


20 


23.5 


86 


21 


24.4 


Total 


678 


106 


15.6 


702 


90 


12.8 


Langford 




October 30, 1990 




October 30, 1991 




Grade 


Enrollment 


Overage % 


Overage 


Enrollment 


Overage % 


Overage 


K 


79 


2 


2.5 


74 


5 


6.8 


1 


78 


11 


14.1 


92 


8 


8.7 


2 


85 


11 


12.9 


82 


14 


17.1 


3 


79 


15 


19.0 


83 


13 


15.7 


4 


72 


20 


27.8 


81 


16 


19.8 


5 


57 


14 


24.6 


67 


17 


25.4 


Total 


510 


73 


14.3 


538 


73 


13.6 


Patton 




October 30, 1990 




October 30, 1991 




Grade 


Enrollment 


Overage % 


Overage 


Enrollment 


Overage % 


Overage 


K 


151 


1 


4.6 


153 


5 


3.3 


1 


190 


25 


13.2 


188 


22 


11.7 


2 


164 


21 


12.8 


185 


24 


13.0 


3 


159 


27 


17.0 


163 


24 


14.7 


4 


160 


10 


6.3 


161 


25 


15.5 


5 


181 


21 


11.6 


158 


12 


7.6 


Total 


1005 


111 


11.0 


1,008 


112 


IIJ 



4. 




91.30 
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Appendix C2: Numuers and Percent of At-Risk Students 



Andrews 





October 30, 1990 




October 30, 1991 




Grade 


enrollment 


At Risk % At Risk 


Enrollment 


At Risk % 


At Risk 


K 


128 


33 


25.8 


142 


41 


28.9 


1 


120 


69 


57.5 


140 


51 


36.4 


2 


118 


42 


35.6 


134 


69 


51.5 


3 


118 


48 


40.7 


113 


72 


63.7 


4 


113 


63 


55.8 


112 


78 


69.6 


5 


95 


60 


63.2 


128 


94 


73.4 


Total 


764 


332 


43.5 


869 


441 


50.7 


Galindo 




October 30, 1990 




October 30, 1991 




Grade 


Enrollment 


At Risk % At Risk 


Enrollment 


At Risk % 


At Risk 


K 


126 


15 


11.9 


120 


23 


18.7 


1 


135 


81 


60.0 


119 


18 


14.4 


2 


102 


26 


25.5 


122 


65 


48.9 


3 


120 


50 


41.7 


90 


50 


52.1 


4 


87 


41 


47.1 


93 


61 


60.4 


5 


94 


42 


44.7 


86 


53 


56.4 


Total 


714 


269 


37.7 


702 


284 


38.2 


Langford 




October 30, 1990 




October 30, 1991 




Grade 


Enrollment 


At Risk % 


At Risk 


Enrollment 


At Risk % 


At Risk 


K 


89 


9 


10.1 


85 


15 


17.6 


1 


79 


39 


49.4 


96 


17 


17.7 


2 


90 


21 


23.3 


85 


42 


49.4 


3 


85 


27 


31.8 


88 


55 


62.5 


4 


78 


33 


42.8 


84 


48 


57.1 


5 


65 


32 


49.2 


74 


44 


59.5 


Total 


546 


175 


32.1 


572 


238 


41.6 


Patten 




October 30, 1990 




October 30, 1991 




Grade 


Enrollment 


At Risk % 


At Risk 


Enrollment 


At Risk % 


At Risk 


K 


152 


9 


5.9 


153 


6 


3.9 


1 


194 


69 


35.6 


190 


29 


15.3 


2 


168 


28 


16.7 


190 


49 


25.8 


3 


167 


36 


21.6 


168 


45 


26.8 


4 


162 


20 


12.3 


172 


58 


33.7 


5 


184 


38 


20.7 


161 


36 


22.4 


Total 


1027 


200 


19.5 


1,034 


223 


21.6 



37 



91.30 
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